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Description 

ROLL STABILITY CONTROL SYSTEM FOR 
AN AUTOMOTIVE VEHICLE USING AN 
EXTERNAL ENVIRONMENTAL SENSING 

SYSTEM 

Background of Invention 

[0001] The present invention relates generally to a dynamic be- 
havior control apparatus for an automotive vehicle, and 
more specifically, to a method and apparatus for control- 
ling the roll characteristics of the vehicle by determining 
roll characteristics using at least in part a camera, radar, 
lidar, or sonar-based system. 

[0002] Dynamic control systems for automotive vehicles have re- 
cently begun to be offered on various products. Dynamic 
control systems typically control the yaw of the vehicle by 
controlling the braking effort at the various wheels of the 
vehicle. Yaw control systems typically compare the desired 
direction of the vehicle based upon the steering wheel an- 



gle and the direction of travel. By regulating the amount 
of braking at each corner of the vehicle, the desired direc- 
tion of travel may be maintained. Typically, the dynamic 
control systems do not address roll of the vehicle. For 
high profile vehicles in particular, it would be desirable to 
control the rollover characteristic of the vehicle to main- 
tain the vehicle position with respect to the road. That is, 
it is desirable to maintain contact of each of the four tires 
of the vehicle on the road. 
[0003] | n vehicle roll stability control it is desired to alter the ve- 
hicle attitude such that its motion along the roll direction 
is prevented from achieving a predetermined limit 
(rollover limit) with the aid of the actuation from the avail- 
able active systems such as controllable brake system, 
steering system and suspension system. Although the ve- 
hicle attitude is well defined, direct measurement is usu- 
ally impossible. 

[0004] There are two types of vehicle attitudes needed to be dis- 
tinguished. One is the so-called global attitude, which is 
sensed by the angular rate sensors. The other is the rela- 
tive attitude, which measures the relative angular posi- 
tions of the vehicle with respect to the road surface on 
which the vehicle is driven. The global attitude of the ve- 



hide is relative to an earth frame (or called the inertia 
frame), sea level, or a flat road. It can be directly related 
to the three angular rate gyro sensors. While the relative 
attitude of the vehicle measures the relative angular posi- 
tions of the vehicle with respect to the road surface, which 
are always of various terrains. Unlike the global attitude, 
there are no gyro-type sensors that can be directly related 
to the relative attitude. A reasonable estimate is that a 
successful relative attitude sensing system utilizes both 
the gyro-type sensors (when the road becomes flat, the 
relative attitude sensing system recovers the global atti- 
tude) and some other sensor signals. 
[0005] one reason to distinguish relative and global attitude is 
due to the fact that vehicles are usually driven on a three- 
dimensional road surface of different terrains, not always 
on a flat road surface. Driving on a road surface with a 
large road bank does increase the rollover tendency, i.e., a 
large output from the global attitude sensing system 
might well imply an uncontrollable rollover event regard- 
less of the flat road driving and the 3-D road driving. 
However driving on a three-dimensional road with moder- 
ate road bank angle, the global attitude may not be able 
to provide enough fidelity for a rollover event to be distin- 



guished. Vehicular rollover happens when one side of the 
vehicle is lifted from the road surface with a long duration 
of time without returning back. If a vehicle is driven on a 
banked road, the global attitude sensing system will pick 
up certain attitude information even when the vehicle 
does not experience any wheel lifting (four wheels are al- 
ways contacting the road surface). Hence a measure of the 
relative angular positions of the vehicle with respect to the 
portion of the road surface on which the vehicle is driven 
provides more fidelity than global attitude to sense the 
rollover event when the vehicle is driven on a road with a 
moderate bank angle. Such an angle is called body- 
to-road roll angle and it is used as one of the key vari- 
ables in the roll stability control module to compute the 
amount of actuation needed for preventing an untripped 
rollover event. 

[0006] when the vehicle does not have one side lifted, U.S. patent 
6,556,908 does provide a method to calculate the relative 
attitudes and their accuracy may be affected by the vehicle 
loading, suspension and tire conditions. However, during 
a potential rollover event, such a relative roll angle is not a 
good measure of the true relative roll angle between vehi- 
cle body and the road surface. U.S. patent application 



10/459,697, filed June 11, 2003 (Attorney Docket No. 
201-0938 (FGT1660)) provides another way to compute 
the true relative roll angle during a potential rollover 
event. This application is suited for cases where vehicle 
loading and suspension conditions are very close to the 
nominal systems. If the vehicle has large loading varia- 
tions (especially roof loading), potential inaccuracy could 
cause false activations in roll stability controls. 

[0007] During a potential rollover event, one or two wheels on 

the inside of the vehicle turn are up in the air and there is 
an angle between the axle of the lifted wheel and road 
surface. Such an angle is called awheel departure angle. If 
such a wheel departure can be somehow characterized, 
the true body-to-road roll angle can be conceptually ob- 
tained as the sum of the wheel departure angle and the 
relative roll angle calculated in U.S. patent 6,556,908. 

[0008] Another way to capture the true body-to-road roll angle is 
to use the resultant angle obtained by subtracting the 
road bank angle for the global roll angle calculated for 
example in U.S. patent 6,631,317. Although this method 
is theoretically feasible, it has inevitable drawbacks. The 
first drawback lies in the computation of the road bank 
angle, since there is no robust and accurate computation 



of road banks using the existing sensor set. Secondly, the 
global roll angle computation as shown in U.S. patent 
6,631,317 may be affected by the accuracy of the low fre- 
quency bank angle estimation. 

[0009] Therefore, the aforementioned two methods of computing 
the body-to-road roll angle may not deliver accurate 
enough body-to-road roll angle for roll stability control 
purpose in certain situations. 

[0010] Various accident avoidance systems are being developed 
that have cameras mounted in the vehicle. Some systems 
also use the camera in determining when to deploy an 
airbag. These systems, however, do not prevent the vehi- 
cle from rolling over. Such systems merely react to a con- 
dition to reduce occupant injury. 

[0011] Therefore, it is desirable in vehicle dynamics control, es- 
pecially for roll stability control to detect accurately vari- 
ous roll angles associated with the vehicle and road to ac- 
curately predict the true roll position of the vehicle to 

properly activate the vehicle control systems. 
Summary of Invention 

[0012] one advantage of the invention is that the camera, radar, 
lidar, or sonar-based systems can be used alone or to- 
gether with various sensors. The camera, radar, lidar, or 



sonar-based systems may provide a check to the sensor 
outputs. 

[0013] other objects and features of the present invention will 

become apparent when viewed in light of the detailed de- 
scription of the preferred embodiment when taken in con- 
junction with the attached drawings and appended claims. 
Brief Description of Drawings 

[0014] Figure 1 is a diagrammatic view of a vehicle with variable 
vectors and coordinated frames according to the present 
invention. 

[0015] Figure 2 is a block diagram of a stability system according 
to the present invention. 

[0016] Figure 3 is an end view of an automotive vehicle on a bank 
with definitions of various angles including global roll an- 
gle, relative roll angle, wheel departure angle (WDA), road 
bank angle and body-to-road angle. 

[0017] Figure 4 is a block diagrammatic view of the controller of 
the present invention. 

[0018] Figure 5 is a block diagrammatic view of the rollover con- 
trol law of Figure 4. 

[0019] Figure 6 is a block diagrammatic view of the handling loss 
control law of Figure 4. 

[0020] Figure 7 is a perspective view of a field of view of a cam- 



era from which image signals are derived. 
Detailed Description 

[0021] | n the following figures the same reference numerals will 
be used to identify the same components. 

[0022] The present invention may be used in conjunction with a 
rollover control system for a vehicle. The system may be 
used with various dynamic control systems such as, but 
not limited to, anti-lock brakes, traction control and yaw 
control systems. The present invention will be discussed 
below in terms of preferred embodiments relating to an 
automotive vehicle moving in a three-dimensional road 
terrain. Further, the various image detection and sensors 
may be used alone or in various combinations depending 
on the conditions. For example, sensors may be used to 
check the image or radar signals or vice versa. 

[0023] Referring to Figures 1, an automotive vehicle 10 with a 
safety system of the present invention is illustrated with 
the various forces and moments thereon during a rollover 
condition. Vehicle 10 has front right and front left tires 
12a and 12b and rear right tires 13a and left rear tires 
13b respectively. The vehicle 10 may also have a number 
of different types of front steering systems 14a and rear 
steering systems 14b including having each of the front 



and rear wheels configured with a respective controllable 
actuator, the front and rear wheels having a conventional 
type system in which both of the front wheels are con- 
trolled together and both of the rear wheels are controlled 
together, a system having conventional front steering and 
independently controllable rear steering for each of the 
wheels, or vice versa. Generally, the vehicle has a weight 
represented as Mg at the center of gravity of the vehicle, 

2 

where g=9.8m/s and M is the total mass of the vehicle. 
[0024] The sensing system 16 may use a standard yaw stability 
control sensor set (including lateral accelerometer, yaw 
rate sensor, steering angle sensor and wheel speed sen- 
sor) together with a roll rate sensor and a longitudinal ac- 
celerometer. Of course, the actual sensors used will vary 
depending on the type of dynamic control system. The 
various possible sensors will be further described below. 
The wheel speed sensors 20 are mounted at each corner 
of the vehicle, and the rest of the sensors of sensing sys- 
tem 16 are preferably mounted directly on the center of 
gravity of the vehicle body, along the directions x, y and z 
shown in Figure 1. As those skilled in the art will recog- 
nize, the frame from b , b and b is called a body frame 

12 3 7 

22, whose origin is located at the center of gravity of the 



car body, with the b corresponding to the x axis pointing 
forward, b> 2 corresponding to the y axis pointing off the 
driving side (to the left), and the b> 3 corresponding to the 
z axis pointing upward. The angular rates of the car body 
are denoted about their respective axes as u> for the roll 

X 

rate, u> for the pitch rate, and co for the yaw rate. The 

y z 

present invention calculations preferably take place in an 
inertial frame 24 that may be derived from the body frame 
22 as described below. 
[0025] As will be described below, the sensing system 16 may 
also include a lidar, radar and/or sonar sensor, cameras 
and/or a GPS system (all of which are shown in Figure 2 
below). 

[0026] The angular rate sensors and the accelerometers are 

mounted on the vehicle car body along the body frame di- 
rections b , b and b , which are the x-y-z axes of the 

l' 2 3 7 

vehicle's sprung mass. 
[0027] The longitudinal acceleration sensor is mounted on the 
car body located at the center of gravity, with its sensing 
direction along b^axis, whose output is denoted as a^. 
The lateral acceleration sensor is mounted on the car 
body located at the center of gravity, with its sensing di- 
rection along b 2 ~axis, whose output is denoted as a^. 



[0028] The other frame used in the following discussion includes 
the road frame, as depicted in Figure 1. The road frame 
system r r r is fixed on the driven road surface, where 

1 1 2 3 

the r 3 axis is along the average road normal direction 
computed from the normal directions of the four- 
tire/road contact patches. 
[0029] | n the following discussion, the Euler angles of the body 
frame b b b with respect to the road frame r r r are de- 

12 3 K 12 3 

noted as 6 ,6 and 6 , which are also called the rela- 

xbr ybr zbr 

tive Euler angles. 

[0030] Referring now to Figure 2, roll stability control system 18 
is illustrated in further detail having a controller 26 used 
for receiving information from a number of sensors which 
may include speed sensors 20, a yaw rate sensor 28, a 
lateral acceleration sensor 32, a roll rate sensor 34, a ver- 
tical acceleration sensor 35, a longitudinal acceleration 
sensor 36, a pitch rate sensor 37, and steering angle po- 
sition sensor 38. Sensors 28-38 may be part of an inte- 
grated measurement unit 40 or IMU. 

[0031] | n one embodiment the sensors are located at the center 
of gravity of the vehicle. Those skilled in the art will rec- 
ognize that the sensor may also be located off the center 
of gravity and translated equivalently thereto. 



[0032] Lateral acceleration, roll orientation and speed may be 
obtained using a global positioning system (GPS) 41. 

[0033] The controller 26 may also be coupled to a lidar, radar, or 
sonar 42. The lidar, radar, or sonar 42 may be used to 
generate a velocity signal of an object. The radar or lidar 
may also be used to generate a trajectory signal of an ob- 
ject. Likewise, the velocity of the vehicle in various direc- 
tions may be obtained relative to a stationary object. A li- 
dar, radar, or sonar sensor 42 may be mounted in various 
positions of the vehicle including the front, sides and/or 
rear. Multiple sensors 42 may also be employed in multi- 
ple locations to provide multiple information from multi- 
ple positions of the vehicle. 

[0034] Controller 26 may also be coupled to a camera system 43 
having cameras 43a-43e. A stereo pair of cameras 42a, 
42b may be mounted on the front of the vehicle to detect 
target objects in front of the vehicle, to measure the ob- 
ject size, range and relative velocity and to classify those 
objects into appropriate categories. Camera 43c may be 
mounted on the right side of the vehicle, camera 43d may 
be mounted on the left side of the vehicle, and camera 
43e may be directed rearward of the vehicle. All or some 
of the cameras may be used in a commercial embodiment. 



Also, a stereo pair of cameras 43a, 43b may be replaced 
by a single camera (43a or 43b) depending on the roll 
condition measured by the system. Various types of cam- 
eras would be evident to those skilled in the art. Various 
types of cameras such as a CMOS-type camera or a CCD- 
type camera may be implemented to generate various im- 
age signals. As will be further described below, the vari- 
ous image signals may be analyzed to determine the vari- 
ous dynamic conditions of the vehicle. 
[0035] Based upon inputs from the sensors and/or cameras, GPS, 
and lidar or radar, controller 26 may control a safety de- 
vice 44. Depending on the desired sensitivity of the sys- 
tem and various other factors, not all the sensors 28-38, 
cameras 43a-43c, lidar or radar 42, or CPS 41 may be 
used in a commercial embodiment. Safety device 44 is 
part of a vehicle subsystem control. Safety device 44 may 
control a passive safety device 46 such as an airbag or a 
steering actuator 48, a braking actuator 50 at one or more 
of the wheels 12a, 12b, 13a, 13b of the vehicle. Engine 
intervention 52 may act to reduce engine power to pro- 
vide a safety function. Also, other vehicle components 
such as a suspension control 54 may be used to adjust 
the suspension to prevent rollover. An anti-roll bar sys- 



tern 56 may be used to prevent rollover. The anti-roll bar 
system 56 may comprise a front or rear active anti-roll 
bar, or both. It should also be noted that the systems 
48-56 may act alone or in various combinations to pre- 
vent the vehicle from rolling over. Certain systems 48-56 
may act to prevent rollover where various dynamic condi- 
tions are sensed. 

[0036] a warning device 60 may also be coupled to controller 26. 
Warning device may warn of various conditions such as an 
impeding rollover or an approach of an in-path object. 
The warnings are provided in time for the driver to take 
evasive action. The warning device may be a visual display 
such as warning lights or an alpha-numeric display such 
an LCD screen. The warning device may also be an audible 
display such as a warning buzzer, chime or bell. The 
warning device may also be a haptic warning such as a vi- 
brating steering wheel. Of course, a combination of audi- 
ble, visual, and haptic display may be implemented. 

[0037] a level-based system 62 may also be coupled to con- 
troller 18. Level-based system 62 uses the pitch level or 
angle of the vehicle to adjust the system. Level-based 
system 62 may, for example, be a headlight adjustment 
system 64 or a suspension leveling system 66. Headlight 



adjustment system 64 adjusts the beam pattern down- 
ward for a loaded vehicle. Suspension leveling system 66 
adjusts the suspension at the various corners of the vehi- 
cle to maintain the vehicle body relatively level. The level- 
based system 62 may also make an adjustment based on 
the roll angle of the vehicle. 

[0038] r 0 || ra te sensor 34 and pitch rate sensor 37 may sense 
the roll condition of the vehicle based on sensing the 
height of one or more points on the vehicle relative to the 
road surface. Sensors that may be used to achieve this in- 
clude a radar-based proximity sensor, a laser-based 
proximity sensor and a sonar-based proximity sensor. 

[0039] Roll rate sensor 34 and pitch rate sensor 37 may also 

sense the roll condition based on sensing the linear or ro- 
tational relative displacement or displacement velocity of 
one or more of the suspension chassis components which 
may include a linear height or travel sensor, a rotary 
height or travel sensor, a wheel speed sensor used to look 
for a change in velocity, a steering wheel position sensor, 
a steering wheel velocity sensor and a driver heading 
command input from an electronic component that may 
include steer by wire using a hand wheel or joy stick. 

[0040] The roll condition may also be sensed by sensing the 



force or torque associated with the loading condition of 
one or more suspension or chassis components including 
a pressure transducer in an act of air suspension, a shock 
absorber sensor such as a load cell, a strain gauge, the 
steering system absolute or relative motor load, the steer- 
ing system pressure of the hydraulic lines, a tire laterally 
force sensor or sensors, a longitudinal tire force sensor, a 
vertical tire force sensor or a tire sidewall torsion sensor. 

[0041] The roll condition of the vehicle may also be established 
by one or more of the following translational or rotational 
positions, velocities or accelerations of the vehicle includ- 
ing a roll gyro, the roll rate sensor 34, the yaw rate sensor 
28, the lateral acceleration sensor 32, a vertical accelera- 
tion sensor, a vehicle longitudinal acceleration sensor, lat- 
eral or vertical speed sensor including awheel-based 
speed sensor, a radar-based speed sensor, a sonar-based 
speed sensor, a laser-based speed sensor or an optical 
based speed sensor. 

[0042] steering control 48 may control the position of the front 
right wheel actuator, the front left wheel actuator, the rear 
left wheel actuator, and the right rear wheel actuator. Al- 
though as described above, two or more of the actuators 
may be simultaneously controlled. For example, in a rack- 



and-pinion system, the two wheels coupled thereto are 
simultaneously controlled. Based on the inputs from sen- 
sors 28 through 38, controller 26 determines a roll condi- 
tion and controls the steering position of the wheels. 

[0043] Speed sensor 20 may be one of a variety of speed sensors 
known to those skilled in the art. For example, a suitable 
speed sensor may include a sensor at every wheel that is 
averaged by controller 26. Preferably, the controller trans- 
lates the wheel speeds into the speed of the vehicle. Yaw 
rate, steering angle, wheel speed and possibly a slip angle 
estimate at each wheel may be translated back to the 
speed of the vehicle at the center of gravity. Various other 
algorithms are known to those skilled in the art. Speed 
may also be obtained from a transmission sensor. For ex- 
ample, if speed is determined while speeding up or brak- 
ing around a corner, the lowest or highest wheel speed 
may not be used because of its error. Also, a transmission 
sensor may be used to determine vehicle speed. 

[0044] Controller 26 may include an integrated roll control sys- 
tem 58 to detect and prevent rollover. While these func- 
tions are provided by controller 26, several controllers 
may be used to provide various determinations and con- 
trol functions. The controller 26 may be programmed to 



provide the various functions. 
[0045] Referring now to Figure 3, the relationship of the various 
angles of the vehicle 10 relative to the road surface 11 is 
illustrated. The present teaching determines a wheel de- 
parture angle 6 , which is the angle from the axle or the 
wheel axis to the road surface 11. Also shown is a refer- 
ence road bank angle 6 , which is shown relative to the 

bank 

vehicle 10 on a road surface. The vehicle 10 has a vehicle 
body 10a and vehicle suspension 10b. The relative roll 
angle 6 is the angle between the wheel axle and the 

xr 

body 10a. The global roll angle 9 is the angle between 
the horizontal plane (e.g., at sea level) and the vehicle 
body 10a. 

[0046] Referring now to Figure 4, controller 26, and in particular 
the integrated roll control system 58, is illustrated in fur- 
ther detail. The integrated roll control system 58 includes 
rollover control law 80 and handling loss control law 82. 
The rollover control law 80 and handling loss control law 
82 are coupled to a roll moment command 84. The 
rollover control law ultimately determines the rollover po- 
tential of the vehicle from the various inputs. The han- 
dling loss control 82 determines a loss of handling of the 
vehicle from the inputs. Handling loss may be wheels 



locking or slipping or the like. The output of the roll mo- 
ment command 84 is used to control the brake pressure 
priority logic 86 and other priority logic 88. The other pri- 
ority logic may include at least one of the steering system 
(actuator) 48, engine intervention 52, suspension control 
54 and anti-roll bar system 56. It should be noted that in 
a simple system only one system such as braking or 
steering may be controlled to prevent rollover performed 
and thus other priority control logic may be eliminated. 
The brake pressure priority logic 86 and the other priority 
control logic 88 receive the roll angle from the roll mo- 
ment command 84 and controls brake control 50 or other 
systems 52-56. The control provided by the brake control 
50 and the other systems are fed back to the system 
through at least one of the roll rate sensor 34, longitudi- 
nal acceleration sensor 36, lateral acceleration sensor 32, 
wheel speed sensors 20, steering angle position sensor 38 
and the camera system 43. The information from these 
systems is fed back to the yaw stability controller 102, an 
anti-lock brake system 104, and a traction control system 
106. The anti-lock brake system 104 generates a brake 
control signal. The traction control system 106 generates 
a brake traction control signal. The yaw stability controller 



102 generates a brake yaw control signal. The yaw stabil- 
ity controller 102 may also provide an input with respect 
to the yawing of the vehicle to the other priority logic 88 
and the brake pressure priority logic 86. The controllers 
102, 104, 106 may be separate devices or integrated into 
controller 26 above. Each controller 102, 104, 106 is part 
of a safety system 44 of Figure 2. The brake pressure pri- 
ority logic 86 receives the brake control signals and gen- 
erates brake controls to prevent the vehicle from rolling 
over. The other systems including but not limited to the 
steering actuator 48, engine intervention 52, suspension 
control 54, and the anti-roll bar system 54 may also be 
controlled and coordinated with the brake control to pre- 
vent the vehicle from rolling over or losing handling. The 
anti-roll bar system, or other systems may be used to si- 
multaneously control rollover while the brake system may 
be used to control yawing or vice-versa after both sys- 
tems are used for rollover control. The anti-roll bars or 
other systems may also be controlled sequentially with the 
brakes to prevent rollover. 
[0047] Referring now to Figure 5, rollover control law 80 is illus- 
trated in further detail. Rollover control law 80 may com- 
prise a rollover detector 110 that is coupled to the various 



sensors. Further, a roll angle computation 112 may also 
be performed by rollover control law 80. The rollover de- 
tector detects the presence of rollover and determines a 
roll angle of the vehicle. Rollover detector may, for exam- 
ple, provide a single wheel lift identification 114 or a dou- 
ble wheel lift identification 116. The output of the single 
wheel lift identification 114, double wheel lift identifica- 
tion 116, and roll angle computation 112 are provided to 
a rollover feedback control computation 118. The rollover 
feedback control computation generates a moment M r 
that is the roll moment of the vehicle. Thus, by knowing 
the roll moment of the vehicle the roll moment can be 
counteracted using one of the systems described below. 
[0048] Referring now to Figure 6, handling roll control law 82 

may also consist of a rollover detector 120 and a roll an- 
gle computation 122. The roll detector in a similar manner 
to the rollover control law 80 may provide an indication of 
single wheel lift detection 124, or double wheel lift detec- 
tion 126. Single wheel lift detection 124, double wheel lift 
detection 126, and roll angle computation 122 may be 
provided to a handling roll feedback control computation 
128 that generates a roll moment M r of the vehicle. By 
knowing the roll angle of the vehicle, the roll moment may 



be counteracted. 
[0049] Referring now to Figure 7, a perspective view of a field of 
view of one or both of the front cameras is illustrated. 
Similar views would be available to the side cameras. Sim- 
ilar views would also be generated from the rear camera. 
Many of the calculations from the front camera may be 
performed by the rear camera. Based on various cues, 
various dynamic conditions such as the roll angle, vehicle 
speed, body-to-road angle, longitudinal and lateral veloc- 
ities, pitch angle, road departure, an in-path object, 
wheels lifting, and body slides that may be determined. 
The front, rear, side, and front or rear and a side camera 
may be used to obtain the points or visual cues described 
below. 

[0050] one such way in which the roll angle between the body 
and the road surface may be estimated is by determining 
a road surface plane 140 by identifying discernable fea- 
tures on the road surface, such as solid lines, dashed 
lines, cats eyes, or other road surface markings or road 
texture. The range to these features may be determined 
based on a stereo vision-based depth map, monocular vi- 
sion-based perspective analysis or matching and scaling 
the observed feature to a pre-defined feature of known 



dimension. Analysis of the change in perspective of the 
defined road surface over time can be used to estimate 
body to road roll angle. 

[0051] The global vehicle longitudinal velocity and lateral velocity 
relative to the lanes can be determined by measuring the 
relative velocity of defined stationary road cues including 
points 142, 144, and 146. 

[0052] The absolute roll angle may be estimated based on analy- 
sis of objects with strong vertical or horizontal structures 
such as street lights 148, bridges, signs or buildings 150. 
Such an algorithm uses multiple possible targets in a fil- 
tering strategy to remove outliers to provide an accurate 
result. Further, the global lateral velocity may be esti- 
mated based on a measurement of range, range rate, an- 
gle, and a change in the stationary structures. 

[0053] a define horizon line may be used to estimate the abso- 
lute roll angle. The horizon 152 is the location in the im- 
age where the road approaches zero width. It can be de- 
termined by detecting the transition from the ground to 
the sky in the image and/or it may be determined by cal- 
culating the point at which the lanes approach zero width. 

[0054] The roll rate may be estimated by measuring the change 
in the roll angle to distant objects such as clouds 154 or 



mountains 156. 

[0055] During night conditions, the illumination pattern on the 
road due to the headlights and parking lights of the vehi- 
cle may be used. Each vehicle has a unique consistent 
characteristic of bright spots, dark spots, and beam cut- 
off. By analyzing the projected pattern it is possible to de- 
termine the road angle between the vehicle body and the 
road surface. 

[0056] Ego-motion may also be analyzed to determine the vehi- 
cle motion. Image values in two sequential images are an- 
alyzed to determine a best fit motion that corresponds to 
the change in the image. The global approach may be very 
robust due to varying road and weather conditions. Global 
roll rate, pitch rate, yaw rate, lateral and longitudinal 
speeds may be determined. Roll pitch and yaw angles may 
also be calculated based on the velocity signals. 

[0057] The vision sensing system may also be used to detect in- 
path objects that may, for example, present a tripping 
hazard for the vehicle. 

[0058] a vision-based, radar-based, or lidar-based sensing sys- 
tem may also be used to detect wet roads, spray and snow 
conditions. 

[0059] Tripping objects may be detected in various manners. 



Tripping objects, such as curbs, are low profile objects 
that if the vehicle slides into may cause a rollover. If the 
vehicle is yawing, the front and side cameras may be 
used. Further, the lateral velocity of the vehicle may be 
determined by the side cameras in relation to various ob- 
jects. Combinations of the side camera and front cameras 
may be used to determine the vehicle side slip. In prior 
systems that use only sensors, thresholds for entering roll 
stability control and yaw stability control are set to take 
into account variability within the sensors. That is, be- 
cause the angles are not measured directly, signal condi- 
tioners are implemented to estimate the road bank and 
vehicle roll angle. By improving the accuracy of the roll 
angle, the thresholds for entry into the yaw stability con- 
trol events can be reduced allowing yaw stability control 
to activate sooner, if necessary. The present invention 
also improves entry into a wheel lift detection algorithm. 
An active wheel lift detection scheme starts to detect that 
the wheels have lifted based on an increase in roll angle. 
The threshold for entry is relatively low due to the indirect 
measurements. When a rollover condition is approaching, 
the system actively determines whether the wheels are 
lifted. Thus, a more accurate determination of the roll an- 



gle of the vehicle may result in improved entry into the 
wheel lift detection strategy. Thus, the wheel lift detection 
entry criteria or lift state can be improved by directly 
measuring the angle between the vehicle body and the 
road surface with a vision sensor. 
[0060] The vision sensing system may also be used to differenti- 
ate between aggressive driving and loss of control condi- 
tions by measuring the lateral velocity relative to the lane 
markings and/or the road edges with a vision sensor, the 
yaw stability control system and the roll stability control 
system may differentiate between aggressive driving and 
potential loss of control conditions. For example, when 
the vehicle is within the lane markings during aggressive 
driving, the yaw stability control and roll stability control 
entry thresholds may be increased. Should the vehicle be 
outside or projected to be outside the vehicle lane mark- 
ings in the near future, the entry thresholds may be re- 
duced. 

[0061] | n a typical stability control system a plurality of wheel 
speed sensors are used to obtain the longitudinal speed 
of the vehicle. However, during wheel lock or wheel slip 
conditions if image data is available from the vision sys- 
tem, a determination may be made as to the longitudinal 



vehicle velocity from the vision system. Also, a radar- 
based sensor can measure the longitudinal speed by mea- 
suring the Doppler shift of signal reflections off of the 
road surface. Also, a lidar-based sensor can measure the 
longitudinal speed by measuring the relative velocity to 
stationary objects. The wheel lock or a heavy wheel slip 
condition may be determined by monitoring the wheel 
speeds. If no output or near zero output is determined 
from one of the plurality of the wheel speed sensors, 
wheel lock may be indicated. If one wheel has a low speed 
relative to the others, wheel slip may be indicated. Thus, 
when wheel slip or wheel lock is not determined, the sys- 
tem may use the output of the wheel speed sensors to de- 
termine the longitudinal speed of the vehicle. When a 
wheel lock or wheel slip condition exists, the wheel speed 
derived from the image, radar, or lidar signals may be 
used as an indication of the longitudinal speed of the ve- 
hicle. 

[0062] The present system may also be used to improve the side 
slip estimation of the vehicle. That is, the image signals 
generated by the cameras may be used to estimate the 
lateral velocity, which in turn may be used in the body 
side slip determination. Typically, side slip is measured 



based on the wheel speeds, the steering wheel angle, the 
lateral acceleration and the yaw rate values. However, side 
slip is difficult to measure accurately in low friction side 
slip conditions such as wet or icy roads. The side slip an- 
gle is the tangent of the ratio of the forward velocity and 
side velocity of the vehicle. Once the side slip angle is de- 
termined using the image signal, the dynamic control sys- 
tem may be controlled. It should also be noted that the 
longitudinal velocity may be obtained using the image, 
radar, or lidar signals or wheel speed sensors as described 
above. 

[0063] The image, radar, or lidar-based signals may also be used 
to estimate a low mu of the vehicle surface. That is, wet or 
snow covered conditions may be estimated using the im- 
age, radar, or lidar-based signals. Vision-based sensors 
can estimate wet or snow covered conditions based on 
road texture, reflectivity, and/or color analysis, road spray 
detection, or water drop patterns on the camera surface. 
Radar and lidar-based sensors can estimate wet or snow 
covered conditions based on analysis of the signal return. 
The controller, based upon the image, radar, or lidar- 
based signals, may generate a low mu identification signal 
of the road surface. The system may also quantify the 



level of mu so as to differentiate wet versus snow-covered 
roads. That is, a first low mu value may be identified as 
well as a second low mu value lower than the first low mu 
value. The second value may correspond to snow whereas 
the first value may correspond to rain. In response to the 
low mu identification signal, a dynamics control system 
may be operated. For example, a roll stability control sys- 
tem or a yaw control system may have the entry thresh- 
olds changed in response to the low mu identification sig- 
nal. 

[0064] The image, radar, or lidar-based signals may also be used 
to determine the presence of an in-path hazard in re- 
sponse to the image signal. The dynamics control system 
may have different strategies based upon whether or not 
an object is in the path or not. The different strategies 
may include the entry criteria for control or the control 
thresholds contained therein. The different strategies and 
entry are vehicle specific and are determined through ve- 
hicle testing. The difference between a first control strat- 
egy and a second control strategy is that more aggressive 
maneuvers may take place in the presence of an in-path 
hazard. Thus, in response to the image, radar, or lidar- 
based signals, a second control strategy may be used 



rather than a first control strategy. The dynamic control 
system is then operated in response to the second control 
strategy. The various types of dynamic control systems 
described above such as anti-lock brakes, traction con- 
trol, roll stability control, and yaw stability control may all 
benefit from the two control strategy approach when an 
in-path hazard is present. 
[0065] Another way in which the image signals may be used is for 
correction of the sensor signals from the integrated mea- 
surement unit 40. One way in which sensor drift may be 
corrected is to use GPS. GPS provides a correction mecha- 
nism every 20 Hz while IMUs provide a high frequency 
signal, for example, in the 143 Hz range. Various angular 
rates such as roll, pitch and yaw rates may be computed 
from the image signals. That is, because the image signals 
may directly measure one of the vehicle attitudes or ve- 
locities of the vehicle, the sensor drift from the integrated 
measurement unit may be periodically corrected. The cor- 
rection may take the form of correcting the drift error of 
the various signals from the integrated measurement unit. 
Thus, corrected sensor signals are formed. The corrected 
sensor signals may then be used to operate the dynamic 
control system. As mentioned above, this correction may 



take place several times a second. For example, correction 
may take place at a rate of 35 Hz. 
[0066] Another way in which the image signals may be used is to 
control the level-based system 62 shown in Figure 2. As 
mentioned above, the level-based system may include a 
headlight adjustment system 64 or a suspension leveling 
system 66. Both of those systems may benefit from a de- 
termination of the pitch angle of the vehicle. Thus, the 
image signals from the vision system may be used by the 
controller 26 to determine a pitch angle estimate. Based 
upon the pitch angle estimate from the image signal, the 
level-based system may be controlled. In addition, the 
image signals may be used to generate a roll angle esti- 
mate. The roll angle estimate together with the pitch an- 
gle estimate may be used to control the level-based sys- 
tem. 

[0067] while the invention has been described in connection with 
one or more embodiments, it should be understood that 
the invention is not limited to those embodiments. On the 
contrary, the invention is intended to cover all alterna- 
tives, modifications, and equivalents, as may be included 
within the spirit and scope of the appended claims. 



